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ABSTRACT 



The Measurement and Evaluat ijn Center conducted two 



validity studies designed to determine the test scores to be used by 
the Division in making decisions about credit by examination at the 
University of Texas Austin (UT Austin) in Biology 302, Cellular and 
Molecular Biology, and Biology 303, Structure and Function of 
Organisms. Seventy-nine summer-term students in Biology 3C2 took the 
UT Austin Test for Credit in Biology as part of their final 
examination. In Biology 303, Structure and Function of Organisms, 68 
students took the UT Austin Test for Credit. Analysis of these 
results and course grades for students resulted in a recommendation 
that 31 be the decision score for students in the C range in Biology 
302 and that 39 be the decision score for students in the C range in 
Biology 303. Scores for A and B grades are also recommended for each 
test. Seven tables present scores and findings. (SLD) 
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VALIDITY STUDIES OF UT AUSTIN TESTS FOR USE IN CREDIT BY 
EXAMINATION IN CELLULAR AND MOLECULAR BIOLOGY (BIO 302) 
AND STRUCTURE AND FUNCTION OF ORGANISMS (BIO 303) 
SUMMER 1991 

Nancy H. Ben6 : Barbara G. Dodd, 
and H. Paul Kelley 

At the request of the Division of Biological Sciences at The University of Texas at Austin, the 
Measurement and Evaluation Center (MEC) conducted two validity studies designed to determine the 
test scores to be used by the Division in making decisions about credit by examination in Biology 302, 
Cellular and Molecular Biology, and in Biology 303, Structure and Function of Organisms, 

During the first summer session of 1S91 , locally-prepared tests were administered to students in 
Biology 302 and in Biology 303 as part of their final examinations. T e st scores were then analyzed in 
relation to student performance, as measured by final grades, in each of the courses. 

BIOLOGY 302 

Method 

Subjects 

Seventy-nine (78) students who were enrolled in Biology 302 during the first summer term 
of 1991 served as subjects. Biology 302 is an introduction to structure and function at cellular 
and subcellular levels; an integrated approach to molecular genetics, metabolism, development, 
evolution, and other life processes. One year of high school chemistry or one semester of college 
chemistry is recommended as preparation for this course. 

Materials 

The UT Austin Test for Credit in Biologv 302 was prepared by faculty members of the Division of 
Biological Sciences. It is a one-hour test designed to cover the content of the course and consists of 
50 multiple-choice items. 

Procedure 

In t e first summer term of 1991, the UT Austin Test for Credit in Biologv 302 was included as a 
part of the final examination in Biology 302. The MEC electronically scanned the test answer sheets, 
calculated Number Correct scores, and delivered those scores to the faculty member responsible for 
evaluating students' performances. Then the relationship between final course grades and test 
scores was analyzed. The correlation coefficient between test scores and course grades was 
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computed as well as the means and standard deviations of those measures of achievement. 
Frequency distributions of the Number Correct Scores (0-50 scale) were crosstabulated with the Final 
Course Grades (0-4 scale), and regression equations were obtained to estimate expected scores on 
the test from the course grades and to estimate the expected grades in the course from the test 
scores. Additionally, analyses were performed to estimate the accuracy of placement that would be 
expected to result from use of each test score value as a possible decision score. A table of possible 
decision scores was prepared for departmental consideration. 

Results 

Tables 1.1 to1.4 were prepared by the MEC to present the results of the validity study to the 
Division of Biological Sciences in August 1991. The analyses were based upon test scores and 
course grades for 79 students. 

Table 1.1 shows the frequency distribution of the test scores (column to left) crosstabulated with 
the five grade groups of F through A (columns 3-7) and for the total group of 79 students (column to 
right). Toward the bottom of the table are (a) the number and percentage of students in each grade 
group and in the total group and (b) the mean test score and standard deviation for each grade group 
and for the total group. At the bottom right are the mean final course grade (2.62) and the standard 
deviation (0.99) of those grades, the mean test score (34.84) and the standard deviation (6.87) of 
those scores, and the coefficient of correlation (.86) between the test scores and the course grades. 
(The value of this coefficient of correlation was inflated because the test scores constituted a part of 
the final examination scores, which in turn constituted a part of the final course grades.) 

Also at the right of the table are the two regression equations used to estimate the Expected 
Scores (bottom row) and the Expected Grades (second column). For example, the minimally 
satisfactory grade of C (2.02) was expected for the group of students with a test score of 30. For the 
group of students who made grades of C , the Expected Score on the test was 31 .14. 

Table 1.2 presents the same descriptive statistics for the total group that was represented in 
Table 1.1, but the five grade groups are collapsed into two academic performance groups: 
Unsatisfactory (grades of Fand D, or 0 and 1 ) and Satisfactory (grades of C, S, and A, or 2, 3, and 4). 
The number and percentage of students in each academic performance group, the mean test score, 
and the standard deviation for each group appear in the bottom two rows of the table. 

Table 1.3 presents the expected accuracies of placement for 11 possible decision scores. At 
the left are the placement categories (possible decision scores). The second and third sets of 
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Table 1.1 

Scores on The UT Austin Test in Biology 302 in Relation To Student Performance 
in Biology 302: Frequency Distributions, Descriptive Statistics, Regression 
Equations, Expected Grades, and Expected Scores 
Summer 1991 
(N - 79) 





Final Grades in Bioloav 302 








n 
u 


1 
1 


o 

c 


a 
O 


A 
*♦ 


Total 


Scores 




p 

r 


n 
u 




D 


A 

M 


N 


47 


4.00 










2 


2 


46 


4.00 




- 






3 


3 


45 


3.88 














44 


3.75 










1 


1 


43 


3.63 










2 


2 


42 


3.51 
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3 


4 


41 


3.38 








3 


1 


4 


40 


3.26 








5 


1 


6 


39 


3.13 








7 
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38 


3.01 
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37 


2.89 






{— 
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36 


2.76 








2 
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35 


2.64 














34 


2.52 
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33 


2.39 








3 
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32 


2.27 








2 
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31 


2.14 
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30 


2.02 




2 
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29 


1.90 




1 




1 
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28 


1.77 




1 
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27 


1.65 
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26 


1.53 




1 








1 


25 


1.40 




2 








3 


24 


1.28 














23-17 


1.15-0.41 


1 


4 


2 






7 


Total 


1 


11 


20 


32 


15 


79 


% 


1% 


14% 


25% 


41% 


19% 


100% 


Mean Score 


17.00 


25.82 


30.70 


37.28 


42.93 


34.84 


Standard Deviation 


0.00 


2.85 


4.41 


3.25 


3.02 


6.87 


Expected Score 


19.24 


25.19 


31.14 


37.10 


43.05 





Expected Grade * 
(Test Score x 
0.1238) - 1.6936 



Expected Score - 
(Final Grade x 
5.9532) + 19.2365 



Mean 


Standard 


Grade 


Deviation 


2.62 


0.99 



Coefficient of 
Correlation 
r = .86 
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Table 1.2 

Scores on The UT Austin Test in Biology 302 in Relation tc Student 
Performance in Biology 302: Combined Frequency 
Distributions and Descriptive Statistics 
Summer 1991 
(N = 79) 





Final Grades in Bioloqy ?02 
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2-4 
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43 




2 


2 




42 




4 


4 
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28 
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27 
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1 




25 
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Mean 


Standard 


Total 


l<£ 


67 


79 
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Deviation 












2.62 


0.99 


% 


15% 


85% 


100% 














Mean Score 


25.08 


36.58 


34.84 




Coefficient of 












Correlation 


Standard Deviation 


3.66 


5.70 


6.87 




r= .86 
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Table 1.3 

Scores on The UT Austin Test for Biology 302 in Relation to 
Student Performance in Biology 302: Possible Decision 
Scores and Corresponding Accuracies of Placement 
Summer 1991 
(N - 79) 



Place- 
ment 
Cateejorv 


Cumulative Numoer 
of Students 


Percent of Students in 
Each Placement Cateqorv 


Overall Accuracy 
of Placement 


Unsatisfactory! 
0,1 (N - 12) 


Satisfactory 
2-4 (N-67) 


Unsatisfactory 


Satisfactory 


Placement 
Accuracy 


Number of 
Students 


%of 
Students 


35-up 
Below 35 


Tnn Mirth O 
I uu niyn v 

Correct 12 


AO C^nrranf 

25 Too Low 


Too Wirth HO/ 

1 00 nign Uvo 

Correct 100% 


bj% uorrect 
37% Too Low 


1 00 High 
Correct 
Too Low 


0 
54 

25 


0% 
68% 

32% 


34- up 
Below 34 


Tnn Winn 0 

i uu niyn w 
Correct 12 


ho uorreci 
21 Too Low 


i oo nign u% 
Correct 100% 


69% Correct 
31% Too Low 


Too High 
Correct 
Too Low 


0 
58 

21 


0% 
73% 
27% 


33- up 
Below 33 


i oo nign u 
Correct 12 


ou uorrect 
17 Too Low 


\ oo High 0% 
Correct 100% 


75% Correct 
25% Too Low 


Too High 
Correct 
Too Low 


0 
62 
17 


0% 
78% 
22% 


32 - up 
Below 32 


Tnn Mirth 0 
1 uu niyn w 

Correct 12 


oo ourreci 
12 Too Low 


i co nign j /o 
Correct 100% 


82% Correct 
18% Too Low 


Too High 
Correct 
Too Low 


0 
67 
12 


0% 
85% 
15% 


31 - up 

Below 31 


Ton Htnh 0 
Correct 12 


%J& UUII O Wl 

9 Too Low 


Tnn Mirth n% 
I uu niyn u /o 

Correct 100% 


0/ /o oorrect 
13% Too Low 


1 00 nign 
Correct 
Too Low 


0 
0 

70 

9 


0% 
89% 
11% 


30 - up 
Below 30 


Too Hinh 2 
Correct 10 


8 Too Low 


Too Mirth 1 7o/ 

i uu nign 1 / /o 
Correct 83% 


000/ r^«i»»>**^ 

oovo 00 r rect 
12% Too Low 


1 00 HKjn 
Correct 
Too Low 


2 
69 
8 


3% 
87% 
10% 


29 - up 
Below 29 


Too High 3 
Correct 9 


61 Correct 
6 Too Low 


Too High 25% 
Correct 75% 


91% Correct 
9% Too Low 


Too High 
Correct 
Too Low 


3 
70 
6 


4% 

*T /O 

89% 
8% 


28 - up 
Below 28 


Too High 4 
Correct 8 


63 Correct 
4 Too Low 


Too High 33% 
Correct 67% 


94% Correct 
6% Too Low 


Too High 
Correct 
Too Low 


4 

71 
4 


5% 
90% 

5% 


27 -up 
Below 27 


Too High 4 
Correct 8 


64 Correct 

3 Too Low 


Too High 33% 
Correct 67% 


96% Correct 
4% Too Low 


Too High 
Correct 

Too Low 


4 

72 
3 


5% 
91% 

4% 


26 - up 
Below 26 


Too High 5 
Correct 7 


64 Correct 
3 Too Low 


Too High 42% 
Correct 58% 


96% Correct 
4% Too Low 


Too High 
Correct 
Too Low 


5 
71 

3 


6% 
90% 
4% 


25 -up 
Below 25 


Too High 7 
Correct 5 


65 Correct 
2 Too Low 


Too High 58% 
Correct 42% 


97% Correct 
3% Too Low 


Too High 
Correct 
Too Low 


7 
70 
2 


9% 
89% 

3% 
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Table 1 .4 

Scores on The UT Austin Test in Biology 302 in Relation to 
Student Performance in Biology 302: Scores Suggested 
by Six Guidelines for Use in Selecting Decision Scores 
Summer 1991 
(N = 79) 



Guideline 



UT Austin 
Biology Test 
Score 



1. 



Expected Score for students whose penormance in course was just minimally 
satisfactory (i.e., students with final grades of C; see Expected Score 
row at bottom of Table 1.1). 



31 



Score for which Expected Grade was just minimally satisfactory (i.e., C; see 
Expected Grade column in Table 1.1). 



30 



3. Score for which percents of errors of students in each academic performance 
category (Unsatisfactory, Satisfactory) were most nearly equal. (See % Too 
High and % Too Low values in middle columns of Table 1 .3.) 



30 



4 . Score for which overall percents of errors were most nearly equal. (See % 
Too High and % Too Low values in last column of Table 1 .3.) 



28 



Score that would have cut off (or held back) approximately the same number of 
students as were in the Unsatisfactory performance group. (See Table 1 .2 for 
number of students in the Unsatisfactory group and the test score that most 
nearly identifies that number of low-scoring students.) 



28 



6. Score that would have maximized overall accuracy of placement. (See number 
Correct in next-to-last column of Table 1 .3.) 



27 



7 



columns give the numbers and percentages of students who t in each of the two academic 
performance categories, would have been placed too high and correctly (Unsatisfactory group) and 
correctly and too low (Satisfactory group) by use of each of the 11 possible decision scores. For 
example, if 30 had been the placement decision score, two of the students (17%) in the 
Unsatisfactory Final Grade category would have been placed too high, while eight of the students 
(12%) in the Satisfactory Final Grade category would be placed too low. The column entries also 
report the numbers and percentages of accurately placed students at each of the score placement 
categories. The last set of columns gives ihe overall accuracy of placement, by number and 
percentage, for both academic performance categories combined. 

Table 4.1 lists six guidelines suggested by various authorities for selecting decision scores for 
use in a program of course placement and credit by examination. Each guideline refers the reader to 
one of the preceding tables. In this validity study, the test scores recommended by the six guidelines 
range from 27 to 31. 



During the August 1991 meeting with the chairperson of the Division of Biological Sciences, 
MEC staff members recommended 31 as the decision score for students in the C range. A score of 
31 corresponds to the Expected Score estimated by the regression procedure for the students who 
earned a grade of C in Biology 302 (see Guideline 1 of Table 1 .4). The Expected Scores of students 
who received Biology 302 final grades of B and A, respectively, are 37 and 43 (see Expected Score 
row at the bottom of Table 1.1). The following score ranges for credit by examination in Biology 302 
were recommended by MEC staff members and approved by the Division of Biological Sciences: 



Discussion and Decision Making 



Test Score Range 



Credit and Letter Grade 



43-50 



Credit with Grade of A 



37-42 



Credit with Grade of B 



31-36 



Credit with Grade of C 
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BIOLOGY 303 

The Measurement and Evaluation Center simultaneously conducted the Biology 302 and 
Biology 303 validity studies and utilized identical procedures when analyzing the data sets for the two 
courses. The following description of method and results of the Biology 303 validity study does not 
repeat some of the explanations contained in the preceding presentation of Biology 302 results. 

Method 

Subjects 

Sixty-eight (68) students who were enrolled in Biology 303 during the first summer term of 1991 
served as subjects. Biology 303 is an introduction to the anatomy, reproduction, physiology, 
development, behavior, and evolution of microbes, plants, and animals. Biology 302 with a grade of at 
least a C is a prerequisite for Biology 303. 

Materials 

The UT Austin Test for Credit in Biology 303 was prepared by faculty members of the Division of 
Biological Sciences. It is a one-hour test designed to cover the content of the course and consists of 
50 multiple-choice items. 

Results 

Tables 2.1 to 2.4 present the results of the validity study; the analyses were based on test 
scores and final course grades tor the 68 students enrolled in Biology 303. 

Table 2.1 shows the mean course grade (2.04) and the standard deviation (1.08) of those 
grades, the mean test score (40.09) and the standard deviation (5.40) of those scores, and the 
coefficient of correlation (.85) between the test scores and the final course grades. 

The six guidelines of Table 2.4 recommend decision scores of 39 and 40. 

Discussion and Decision Making 

During the August 1991 meeting with the chairperson of the Division of Biological Sciences, 
MEC staff members recommended 39 as the decision score for students in the C range. A score of 
39 represents the Expected Score of students who earned grades of approximately C- in Biology 
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303, and 43 and 47 represent the Expected Scores of students who received grades of 
approximately S-and respectively. The following score ranges for credit by examination in 
Biology 303 were recorr.mended by MEC staff members and approved by the Division of Biological 
Sciences: 

Test Score Range Credit and Letter Grade 

47-50 Credit with Grade of A 

43-46 Credit with Grade of B 

39-42 Credit with Grade of C 



n 



10 



Table 2.1 

Scores on The UT Austin Test in Biology 303 in Relation To Student Performance 
in Biology 303: Frequency Distributions, Descriptive Statistics, Regression 
Equations, Expected Grades, and Expected Scores 
Summer 1991 
(N - 68) 





Final Grades in Biotoav 303 




Test 


Expected 


0 


1 


2 


3 


4 


Total 


Scores 


Grades 


F 


D 


C 


B 


A 


N 


49 


3.57 










1 


i 


48 


3.40 • 








1 


2 


O 


47 


3.23 








1 


1 


C 


46 


3.05 








4 


1 


c 

o 


45 


2.88 








3 




o 


44 


2.71 






2 


2 




A 


43 


2.54 






2 


3 




5 


42 


2.37 




1 


5 


3 




Q 


41 


2.20 




1 


3 


1 




5 


40 


2.03 






4 


1 




c 

•j 


39 


1.86 




1 


6 






7 


38 


1.69 




2 








o 
c 


37 


1.52 




1 








1 


36 


1.35 




2 








2 


35 


1.18 














34 


1.01 


2 


2 


1 






5 


33 


0.84 




1 


1 






2 


32 


0.66 


2 


1 


1 






4 


31 


0.49 














30 


0.32 














29 


0.15 














28- 15 


0.00-0.00 


3 










3 


Total 


7 


12 


25 


19 


5 


68 


% 


10% 


18% 


37% 


28% 


7% 


100% 


Mean Score 


30.00 


36.67 


40.00 


44.11 


47.60 


40.09 


Standard Deviation 


3.59 


2.98 


3.02 


2.10 


1.02 


5.40 


Expected Score 


31.44 


35.67 


39.90 


44.13 


48.36 





Expected Grade » 
(Test Score x 
0.1707) -4.7976 



Expected Score « 
(Final Grade x 
4.2316U 31.4384 



Mean 


Standard 


Grade 


Deviation 


2.04 


1.08 



Coefficient of 
Correlation 
L=J5 
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Table 2.2 



Scores on The UT Austin Test in Biology 303 in Relation to Student 
Performance in Biology 303: Combined Frequency 
Distributions and Descriptive Statistics 
Summer 1991 
(N = 68) 





Final Grades in Bioloqy 303 




Test 


Unsatisfactory 


Satisfactory 


Total 


Scores 


0,1 


2-4 


N 


49 




1 


1 


48 




3 


3 


47 




2 


2 


46 




5 


5 


45 




3 


3 


44 




4 


4 


43 




5 


5 


42 


1 


8 


9 


41 


1 


4 


5 


40 




5 


5 


39 


1 


6 


7 


38 


2 




2 


37 


1 




1 


36 


2 




2 


35 








34 


4 


1 


5 


33 


1 


1 


2 


32 


3 


1 


4 


31 








30 








29 








28-25 


3 




3 


Total 


19 


49 


68 


% 


23% 


72% 


100% 


Mean Score 


34.21 


42.37 


40.09 


Standard Deviation 


4.55 


3.65 


5.40 



Mean 


Standard 


Grade 


Deviation 


2.04 


1.08 



Coefficient of 
Correlation 
r= .85 
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Table 2.3 

Scores on The UT Austin Test for Biology 303 in Relation to 
Student Performance in Biology 303: Possible Decision 
Scores ana Corresponding Accuracies of Placement 
Summer 1991 
(N - 68) 



Place- 


Cumulative Number 
of Students 


Percent of Students in 
Each Placement Cateqorv 


Overall Accuracy 
of Placement 


ment 

wet ICTVJUl y 


Unsatisfactory 
0.1 (N-191 


Satisfactory 
2-4 (N ~ 49) 


Unsatisfactory 


Satisfactory 


Placement 
Accuracy 


Number of 

Students 


%of 
Students 


43- up 

DolOW *K3 


Too High 0 
Correct 19 


23 Correct 

26 Too Low 


Too High 0% 
Correct 100% 


47% Correct 
53% Too Low 


Too High 
Correct 
Too Low 


0 
42 
26 


0% 
62% 
38% 


42 -up 

DBIOW **c 


Too High 1 
Correct 18 


31 Correct 
18 Too Low 


Too Hi^h 5% 
Correct 95% 


63% Correct 
37% Too Low 


Too High 
Correct 
Too Low 


1 

49 
18 


1% 
72% 
26% 


41 - up 
below *n 


Too High 2 
Correct 1 7 


35 Correct 
14 Too Low 


Too High 11% 
Correct 89% 


71% Correct 
29% Too Low 


Too High 
Correct 
Too Low 


2 
52 
14 


3% 
76% 
21% 


40 - up 
Below 40 


Too High 2 
Correct 17 


40 Correct 
9 Too Low 


Too High 11% 
Correct 89% 


82% Correct 
18% Too Low 


Too High 
Correct 
Too Low 


2 
57 
9 


3% 
84% 
13% 


39- up 
below oa 


Too High 3 
Correct 1 6 


46 Correct 
3 Too Low 


Too High 16% 
Correct 84% 


94% Correct 
6% Too Low 


Too High 
Correct 
Too Low 


3 
62 
3 


4% 
91% 
4% 


38 - up 
Below 38 


Too High 5 
Correct 1 4 


46 Correct 
3 Too Low 


Too High 26% 
Correct 74% 


94% Correct 

6% Too Low 


Too High 
Correct 
Too Low 


5 
60 

3 


7% 
88% 
4% 


37- up 


Too High 6 
Correct 13 
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